{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-title"
    ]
   },
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some output activations to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] [Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012](https://arxiv.org/abs/1207.0580)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "pdf-ignore"
    ]
   },
   "outputs": [],
   "source": [
    "# As usual, a bit of setup\n",
    "from __future__ import print_function\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cs231n.classifiers.fc_net import *\n",
    "from cs231n.data_utils import get_CIFAR10_data\n",
    "from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from cs231n.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "pdf-ignore"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train:  (49000, 3, 32, 32)\n",
      "y_train:  (49000,)\n",
      "X_val:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.items():\n",
    "  print('%s: ' % k, v.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `cs231n/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.25\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  10.014059116977283\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.749784\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.4\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  9.977917658761159\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.600796\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.7\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  9.987811912159426\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.30074\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "for p in [0.25, 0.4, 0.7]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print('Running tests with p = ', p)\n",
    "  print('Mean of input: ', x.mean())\n",
    "  print('Mean of train-time output: ', out.mean())\n",
    "  print('Mean of test-time output: ', out_test.mean())\n",
    "  print('Fraction of train-time output set to zero: ', (out == 0).mean())\n",
    "  print('Fraction of test-time output set to zero: ', (out_test == 0).mean())\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `cs231n/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  5.44560814873387e-11\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.2, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "# Error should be around e-10 or less\n",
    "print('dx relative error: ', rel_error(dx, dx_num))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 1:\n",
    "What happens if we do not divide the values being passed through inverse dropout by `p` in the dropout layer? Why does that happen?\n",
    "\n",
    "## Answer:\n",
    "Train result can't match test result.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `cs231n/classifiers/fc_net.py`, modify your implementation to use dropout. Specifically, if the constructor of the network receives a value that is not 1 for the `dropout` parameter, then the net should add a dropout layer immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  1\n",
      "Initial loss:  2.3004790897684924\n",
      "W1 relative error: 1.48e-07\n",
      "W2 relative error: 2.21e-05\n",
      "W3 relative error: 3.53e-07\n",
      "b1 relative error: 5.38e-09\n",
      "b2 relative error: 2.09e-09\n",
      "b3 relative error: 5.80e-11\n",
      "\n",
      "Running check with dropout =  0.75\n",
      "Initial loss:  2.302371489704412\n",
      "W1 relative error: 1.90e-07\n",
      "W2 relative error: 4.76e-06\n",
      "W3 relative error: 2.60e-08\n",
      "b1 relative error: 4.73e-09\n",
      "b2 relative error: 1.82e-09\n",
      "b3 relative error: 1.70e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.3042759220785896\n",
      "W1 relative error: 3.11e-07\n",
      "W2 relative error: 1.84e-08\n",
      "W3 relative error: 5.35e-08\n",
      "b1 relative error: 5.37e-09\n",
      "b2 relative error: 2.99e-09\n",
      "b3 relative error: 1.13e-10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [1, 0.75, 0.5]:\n",
    "  print('Running check with dropout = ', dropout)\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print('Initial loss: ', loss)\n",
    "  \n",
    "  # Relative errors should be around e-6 or less; Note that it's fine\n",
    "  # if for dropout=1 you have W2 error be on the order of e-5.\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a keep probability of 0.25. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "(Iteration 1 / 125) loss: 7.856644\n",
      "(Epoch 0 / 25) train acc: 0.260000; val_acc: 0.184000\n",
      "(Epoch 1 / 25) train acc: 0.416000; val_acc: 0.258000\n",
      "(Epoch 2 / 25) train acc: 0.482000; val_acc: 0.276000\n",
      "(Epoch 3 / 25) train acc: 0.532000; val_acc: 0.277000\n",
      "(Epoch 4 / 25) train acc: 0.600000; val_acc: 0.271000\n",
      "(Epoch 5 / 25) train acc: 0.708000; val_acc: 0.299000\n",
      "(Epoch 6 / 25) train acc: 0.722000; val_acc: 0.282000\n",
      "(Epoch 7 / 25) train acc: 0.832000; val_acc: 0.255000\n",
      "(Epoch 8 / 25) train acc: 0.878000; val_acc: 0.269000\n",
      "(Epoch 9 / 25) train acc: 0.902000; val_acc: 0.275000\n",
      "(Epoch 10 / 25) train acc: 0.888000; val_acc: 0.261000\n",
      "(Epoch 11 / 25) train acc: 0.928000; val_acc: 0.276000\n",
      "(Epoch 12 / 25) train acc: 0.962000; val_acc: 0.303000\n",
      "(Epoch 13 / 25) train acc: 0.962000; val_acc: 0.306000\n",
      "(Epoch 14 / 25) train acc: 0.968000; val_acc: 0.304000\n",
      "(Epoch 15 / 25) train acc: 0.970000; val_acc: 0.278000\n",
      "(Epoch 16 / 25) train acc: 0.988000; val_acc: 0.301000\n",
      "(Epoch 17 / 25) train acc: 0.982000; val_acc: 0.305000\n",
      "(Epoch 18 / 25) train acc: 0.984000; val_acc: 0.299000\n",
      "(Epoch 19 / 25) train acc: 0.990000; val_acc: 0.301000\n",
      "(Epoch 20 / 25) train acc: 0.982000; val_acc: 0.307000\n",
      "(Iteration 101 / 125) loss: 0.049906\n",
      "(Epoch 21 / 25) train acc: 0.964000; val_acc: 0.310000\n",
      "(Epoch 22 / 25) train acc: 0.988000; val_acc: 0.317000\n",
      "(Epoch 23 / 25) train acc: 0.978000; val_acc: 0.309000\n",
      "(Epoch 24 / 25) train acc: 0.970000; val_acc: 0.302000\n",
      "(Epoch 25 / 25) train acc: 0.982000; val_acc: 0.301000\n",
      "\n",
      "0.25\n",
      "(Iteration 1 / 125) loss: 17.318478\n",
      "(Epoch 0 / 25) train acc: 0.230000; val_acc: 0.177000\n",
      "(Epoch 1 / 25) train acc: 0.378000; val_acc: 0.243000\n",
      "(Epoch 2 / 25) train acc: 0.402000; val_acc: 0.254000\n",
      "(Epoch 3 / 25) train acc: 0.502000; val_acc: 0.276000\n",
      "(Epoch 4 / 25) train acc: 0.528000; val_acc: 0.298000\n",
      "(Epoch 5 / 25) train acc: 0.562000; val_acc: 0.296000\n",
      "(Epoch 6 / 25) train acc: 0.626000; val_acc: 0.291000\n",
      "(Epoch 7 / 25) train acc: 0.622000; val_acc: 0.297000\n",
      "(Epoch 8 / 25) train acc: 0.690000; val_acc: 0.313000\n",
      "(Epoch 9 / 25) train acc: 0.712000; val_acc: 0.296000\n",
      "(Epoch 10 / 25) train acc: 0.722000; val_acc: 0.305000\n",
      "(Epoch 11 / 25) train acc: 0.764000; val_acc: 0.305000\n",
      "(Epoch 12 / 25) train acc: 0.770000; val_acc: 0.290000\n",
      "(Epoch 13 / 25) train acc: 0.828000; val_acc: 0.306000\n",
      "(Epoch 14 / 25) train acc: 0.794000; val_acc: 0.344000\n",
      "(Epoch 15 / 25) train acc: 0.850000; val_acc: 0.335000\n",
      "(Epoch 16 / 25) train acc: 0.832000; val_acc: 0.299000\n",
      "(Epoch 17 / 25) train acc: 0.840000; val_acc: 0.294000\n",
      "(Epoch 18 / 25) train acc: 0.864000; val_acc: 0.320000\n",
      "(Epoch 19 / 25) train acc: 0.866000; val_acc: 0.317000\n",
      "(Epoch 20 / 25) train acc: 0.860000; val_acc: 0.314000\n",
      "(Iteration 101 / 125) loss: 4.255514\n",
      "(Epoch 21 / 25) train acc: 0.908000; val_acc: 0.309000\n",
      "(Epoch 22 / 25) train acc: 0.898000; val_acc: 0.295000\n",
      "(Epoch 23 / 25) train acc: 0.918000; val_acc: 0.295000\n",
      "(Epoch 24 / 25) train acc: 0.914000; val_acc: 0.310000\n",
      "(Epoch 25 / 25) train acc: 0.922000; val_acc: 0.329000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "np.random.seed(231)\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [1, 0.25]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print(dropout)\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver\n",
    "  print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4gAAAJNCAYAAABz4lt9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XucnGV5+P/PZVhkVcxyiEiygQRJg5gEYhcQoYBECCiBgBrBA8WqiELpr4fU0G+bb8y3Fkqo2BRaGxUBD0CKMQYFo+KpeEA2DW4EjBw1u8EaDomoC4Zw/f6Y2TBZdpPdZGee2dnP+/Xa18xzP/c89zW7TyZ77X2KzESSJEmSpBcVHYAkSZIkqT6YIEqSJEmSABNESZIkSVKZCaIkSZIkCTBBlCRJkiSVmSBKkiRJkgATREmSiIhREfHbiDig6FgkSSpSuA+iJGm4iYjfVhy+BHgG2FI+/kBmfr72UUmSNPyZIEqShrWIeAR4X2Z+czt1dsvMZ2sXVe008nuTJNWeQ0wlSQ0nIv4xIm6KiBsi4ingXRFxdET8KCI2RsSjEbE4IprK9XeLiIyICeXjz5XP3xYRT0XEDyNiYj9tvSgibo6IX5Wv/Z2IeHXF+ZdExJUR8cuI2BQR34uIF5fPHVeOaVNErIuId5fL74iI8yqu8b6I+E6vWD8UEQ8APyuXXxURnRHxm4i4KyJeX/H63SLiHyLiwfL59ogYGxH/GRH/3Ov93BYRF+36T0GSNByZIEqSGtWZwBeA0cBNwLPAXwD7AscApwAf2M7r3wH8A7A38Evg/22n7leAScArgZ8Cn604dyUwDTiqfK2/A54rJ5xfBT4G7ANMB9YM4v2dDhwBTC0f31luZ2/gZuC/ehJRYC7wVkrvuQV4H/A0cB3wjoh4EUBE7AccD9w4iDgkSQ3EBFGS1KjuyMxbMvO5zOzOzLsy887MfDYzHwKWUEqG+nNzZrZn5mbg88DhfVUqX//azHwqM58GFgB/HBEvjYhRwHnAxZn5aGZuycw7ytd8F/C1zFxajumxzLx7EO/vnzLzyczsLsfx2cx8ojzc9HLg5cDB5brvA/4uM+8vx3t3ue4PgO6K78M5wDcz87FBxCFJaiAmiJKkRrWu8iAiDomIr5aHgv4GWEipN7E/v6p4/nvgZX1VKq+AenlEPFS+7gPlU/sC+wG7Aw/28dLx/ZQPVO/397cR8bOI2AQ8CbyU59/f9tq6nlKySvnxs/3UkySNACaIkqRG1XsVtv+kNPzz4Mx8OTAfiCFo51zgTcCJlIaz9vTaBfC/wB+AV/XxunX9lAP8jtLqrD1e2Uedre8vIt4A/BXwFkpDSPcCfsvz7297bX0WOCsippfr3NJPPUnSCGCCKEkaKfYENgG/Ky8is735h4O97jPA45SSuo/2nMjMLcC1wMcj4pXl3sZjyovjfA44JSLeUl5EZt+IOKz80ruBt0REc0T8EfBnA4jhWeAxoInSMNeXVpz/FPCPEfGqKDk8IvYux/iLcnvXAf9VHiYrSRqhTBAlSSPFXwN/CjxFqTfxpiG67meA9eWve4Af9Dr/l8B9wCrgCeCfKG0z9TAwC/hwufx/eH7BmSso9RD+GriGUjK5PbcC3wTuBx4BfgM8WnF+EbAcuL18bgmwR8X568ptO7xUkkY490GUJGmEi4gTgU8DB6W/GEjSiGYPoiRJI1hE7E5p+49PmhxKkkwQJUkaoSJiKqUVT/cGFhccjiSpDjjEVJIkSZIE2IMoSZIkSSozQZQkSZIkAbBb0QHUwr777psTJkwoOgxJkiRJKsSqVasey8wxO6o3IhLECRMm0N7eXnQYkiRJklSIiPjFQOo5xFSSJEmSBJggSpIkSZLK6ipBjIhrIuLXEfHTfs5HRCyOiAcioiMiXlvrGCVJkiSpUdVVgghcC5yynfOnApPKX+cD/1GDmCRJkiRpRKirBDEzvwc8sZ0qZwDXZ8mPgJaI2L820UmSJElSY6urBHEAxgHrKo47y2WSJEmSpF003La5iD7Kss+KEedTGobKAQccUM2YJEmSNEDLV3exaOVa1m/sZmxLM3NnTmb2dP/eL9WL4ZYgdgLjK45bgfV9VczMJcASgLa2tj6TSEmSJDBpqZXlq7u4ZNkaujdvAaBrYzeXLFsD4Pe7CryvtTOGW4K4ArgoIm4EjgI2ZeajBcckSZKGsSKTliJ/gS+i7UUr1279Pvfo3ryFRSvX1uR9j6SEyWS89hrl/qqrOYgRcQPwQ2ByRHRGxHsj4oKIuKBc5VbgIeAB4JPAhwoKVZIkNYjtJS3V1PMLfNfGbpLnf4Ffvrqrqu0W2fb6jd2DKh9KRX6/i1DUfT1SNdL9VVc9iJl5zg7OJ3BhjcKRJEkjQFFJS5G9aUW1Pbalma4+vq9jW5qr1maPonsva63IZBwapzdtoBrp/qqrHkRJkqRa6y85qXbSUuQv8EW1PXfmZJqbRm1T1tw0irkzJ1e1XSg+Yaq1ou5raKzetIFqpPvLBFGSJG1j+eoujrnsW0yc91WOuexbNfulrqh2i0paivwFvqi2Z08fx6VnTWVcSzMBjGtp5tKzptakh6XI73cRikzGixzeWtTnSCPdXyaIkiRpq6L+8l9kj0NRSUuRv8AX2fbs6eP4/rwTefiyN/P9eSfWbPhdke+5CEUm40X1phX5OdJI91ddzUGUJKneuMpkbebRFD1/Z/b0cTWfJ9TTXhH3V5FtF6XI91zU50gR9zUUN9e0yM+RRvo3ZYIoSVI/it7+oIi2i/rLfyPN3xmMon6BL7rtohTxnkfidhNzZ07e5j1DbXrTiv4caZR/Uw4xlSSpH0XOoymq7aLm0TTS/B2p0kjcbqKo4a1+jgwNE0RJkvrhKpMltfjLfyPN35EqFd2rVZQi5pr6OTI0TBAlSeqHq0zW7i//RS6oIVWTvVq14+fI0IjS3vONra2tLdvb24sOQ5I0zPSeOwSlv0bX4heOItuWNHT8t6x6ERGrMrNtR/VcpEaSpH64yqSkXeW/ZQ039iBKkiRJUoMbaA+icxAlSZIkSYAJoiRJkiSpzDmIkqRhYfnqLufwSJJUZSaIkqS613sVwK6N3VyybA2ASaIkSUPIIaaSpLq3aOXabZaIB+jevIVFK9cWFJEkSb10LIUrp8CCltJjx9KiI9op9iBKkure+o3dgyqXJBWsYyncvhA2dcLoVpgxH6bNKTqq6ulYCrdcDJvL/y9tWlc6hmH3vu1BlCTVvbEtzYMqlyQVqCdZ2rQOyOeTpVr0qBXVi3f7wueTwx6bu0vlw4wJoiSp7s2dOZnmplHblDU3jWLuzMkFRSRJ6ldRyVKRiemmzsGV1zETRElS3Zs9fRyXnjWVcS3NBDCupZlLz5rqAjWShocGmZs2YEUlS0X24o1uHVx5Hau7OYgRcQrwr8Ao4FOZeVmv8wcC1wBjgCeAd2Xm8EvNJWkXjMQtH2ZPH9fw71FSA2qguWkDNrq13IvXR3k1FdmLN2P+tj9ngKbmUvkwU1c9iBExCrgaOBU4FDgnIg7tVe0K4PrMnAYsBC6tbZSSVKyeLR+6NnaTPL/lw/LVXUWHJknqrcheraJ6LmfMLyVHlWqRLBXZizdtDsxaDKPHA1F6nLV4WP4RoN56EI8EHsjMhwAi4kbgDODeijqHAn9Zfv5tYHlNI5Skgm1vywd72CSpzhTVq1Vkz2XP9Wu9imnRvXjT5gzLhLC3eksQxwGV/dGdwFG96vwEeAulYahnAntGxD6Z+XhtQpSkYrnlgyQNI0UNt9xez2UtkpgikqWiEtMGU28JYvRRlr2O/wa4KiLOA74HdAHPvuBCEecD5wMccMABQxulJBVobEszXX0kg7XY8mEkzn2UpF1SVK9WA62qOSgN0otXpLqag0ipx3B8xXErsL6yQmauz8yzMnM68H/KZZt6Xygzl2RmW2a2jRkzppoxS1JNFbXlg3MfJWknFDU3rYFW1VRt1VsP4l3ApIiYSKln8GzgHZUVImJf4InMfA64hNKKppI0YvT02NW6J8+5j5K0k4ro1Sp6Pp6GrbpKEDPz2Yi4CFhJaZuLazLznohYCLRn5grgBODSiEhKQ0wvLCxgSSNakcMti9jywbmPkjSMOB9PO6muEkSAzLwVuLVX2fyK5zcDN9c6Lkmq1DPcsqdHrWe4JdCwvWlFzn2UJO0E5+NpJ9TbHERJGha2N9yyURU191GSJNVO3fUgStJwMBKHWxY191GSJNWOCaIk7YSROtyyiLmPkqqkY2kx89OKalfSgDjEVJJ2gsMtJQ1rHUtLK1xuWgdk6fGWi0vljdhuZftXToEFLaXHWrUrDSMmiJK0E2ZPH8elZ01lXEszAYxraebSs6bauyZpeLh94bbbH0Dp+PaFjdkuFJ+cSsOEQ0wlaSc53FLSsLWpc3Dlw71d2H5y6hBXaSt7ECVJUn1w+F/tjG4dXPlwbxeKTU6lYcQEUZIkFc/hf7U1Yz409VpUq6m5VN6I7UKxyak0jJggSpKk4hU5N20kmjYHZi2G0eOBKD3OWlz9oZZFtQvFJqfSMOIcREmSVLyROvyvyC0fps0pZu5dke2CW2xIO2CCKEmSije6tTy8tI/yRtUzrLan57RnWC2YtFRLUcmpNIw4xFTSsLd8dRfHXPYtJs77Ksdc9i2Wr+4qOiRJg1X08L8iFshxWK2kOmQPoqRhbfnqLi5ZtobuzVsA6NrYzSXL1gC4BYU0nBQ5/K+onryROqxWUl0zQZQ0rC1auXZrctije/MWFq1ca4Ko4a/I+WlFKGr4X1H7443EYbWS6p5DTCUNa+s3dg+qXBo23PahdorqySt6WK0k9cEEUdKwNraleVDl0rDh/LTaKWp/vCK3fJCkfjjEVNKwNnfm5G3mIAI0N41i7szJBUYlDQHnp9XOjPnbzkGE2vXkuaqmpDpjD6KkYW329HFcetZUxrU0E8C4lmYuPWuq8w81/BXVqzUS2ZMnSVtFZhYdQ9W1tbVle3t70WFIkjRwvVfWhFKvlomLJGknRMSqzGzbUT17ECVJqkf2akmSClB3cxAj4hTgX4FRwKcy87Je5w8ArgNaynXmZeatNQ9UkqRqc36aJKnG6qoHMSJGAVcDpwKHAudExKG9qv09sDQzpwNnA/9e2yglSZIkqTHVVYIIHAk8kJkPZeYfgBuBM3rVSeDl5eejgfU1jE+SpMbXsRSunAILWkqP7r0oSSNGvQ0xHQesqzjuBI7qVWcB8PWI+HPgpcAbaxOaJEkjQO/FcTatKx2Dw10laQSotx7E6KOs9zKr5wDXZmYr8CbgsxHxgvcREedHRHtEtG/YsKEKoUqS1IBuX7jtyqlQOr59YTHxSJJqqt4SxE5gfMVxKy8cQvpeYClAZv4Q2APYt/eFMnNJZrZlZtuYMWOqFK4kSQ1mU+fgyiVJDaXeEsS7gEkRMTEidqe0CM2KXnV+CcwAiIhXU0oQ7SKUJGkojG4dXLkkqaHUVYKYmc8CFwErgfsorVZ6T0QsjIjTy9X+Gnh/RPwEuAE4LzN7D0OVJEk7Y8Z8aGretqypuVQuSWp49bZIDeU9DW/tVTa/4vm9wDG1jkuSpBGhZyGa2xeWhpWObi0lhy5QI0kjQt0liJIk1ZWOpSMvWZo2p/HfoySpTyaIkiT1xy0fJEkjTF3NQZQkqV9FbN7ulg+SpBHGHkRJUv0rqifPLR8kSSOMPYiSpPpXVE+eWz5IkkYYE0RJUv0rqifPLR8kSSOMCaIkqf4V1ZM3bQ7MWgyjxwNRepy12AVqJEkNyzmIkqT6N2P+tnMQoXY9eW75IEkaQexBlCTVP3vyJEmqCXsQJUnDgz15kiRVnT2IkiRJkiTABFGSJEmSVGaCKEmSJEkCTBAlSZIkSWUuUiNpSCxf3cWilWtZv7GbsS3NzJ05mdnTxxUdloZax1K4fWFpg/rRraVtJlw4RpKkhmGCKGmXLV/dxSXL1tC9eQsAXRu7uWTZGgCTxEbSsXTbvQg3rSsdg0miJEkNwiGmknbZopVrtyaHPbo3b2HRyrUFRaSquH3hthvVQ+n49oXFxCNJkoacCaKkXbZ+Y/egyjUEOpbClVNgQUvpsWNp9dvc1Dm4ckmSNOw4xFRqMEXMBRzb0kxXH8ng2JbmqrY7YhU11HN0a6mtvsolSVJDqLsexIg4JSLWRsQDETGvj/NXRsTd5a+fR8TGIuKU6lHPXMCujd0kz88FXL66q6rtzp05meamUduUNTeNYu7MyVVtd8QqaqjnjPnQ1Cvpb2oulUuSpIZQVwliRIwCrgZOBQ4FzomIQyvrZOZfZubhmXk48G/AstpHKtWnouYCzp4+jkvPmsq4lmYCGNfSzKVnTXWBmmopaqjntDkwazGMHg9E6XHWYheokSSpgdTbENMjgQcy8yGAiLgROAO4t5/65wD/t0axSXWvyLmAs6ePMyGslSKHek6bY0IoSVIDq6seRGAcUPlbT2e57AUi4kBgIvCtGsQlDQv9zflzLmCDcainJEmqknpLEKOPsuyn7tnAzZm5pa+TEXF+RLRHRPuGDRuGLECpnjkXsMaKWEkUHOopSZKqpt6GmHYC4yuOW4H1/dQ9G7iwvwtl5hJgCUBbW1t/SabUUHqGeNZ6FdMRqehN4x3qKUmSqiAy6yd3iojdgJ8DM4Au4C7gHZl5T696k4GVwMQcwBtoa2vL9vb2KkQsacS6cko/8wDHw1/+tPbxSJIkbUdErMrMth3Vq6shppn5LHARpeTvPmBpZt4TEQsj4vSKqucANw4kOZSkqnDTeEmS1IDqbYgpmXkrcGuvsvm9jhfUMiZJegE3jZckSQ2oKj2IEXFRROxVjWtLUl1wJVFJktSAqjXE9JXAXRGxNCJOiYi+VieVpOHLlUQlSVIDqtoiNeWk8GTgPUAbsBT4dGY+WJUGt8NFaiRJkiSNZIUvUlNeQOZX5a9ngb2AmyPi8mq1KUmSJEnaeVVZpCYiLgb+FHgM+BQwNzM3R8SLgPuBv61Gu5IkSZKknVetVUz3Bc7KzF9UFmbmcxFxWpXalCRJkiTtgmoNMb0VeKLnICL2jIijADLzviq1KUmSJEnaBdVKEP8D+G3F8e/KZZIkSZKkOlWtBDGyYnnUzHyO6g1nlSRJkiQNgWoliA9FxMUR0VT++gvgoSq1JUmSJEkaAtVKEC8AXg90AZ3AUcD5VWpLkiRJkjQEqjLsMzN/DZxdjWtLkiRJkqqjWvsg7gG8F3gNsEdPeWb+WTXakyRJkiTtumoNMf0s8EpgJvBdoBV4qkptSZIkSZKGQLUSxIMz8x+A32XmdcCbgalVaktSPehYCldOgQUtpceOpUVHJEmSpEGq1tYTm8uPGyNiCvArYEKV2pJUtI6lcMvFsLm7dLxpXekYYNqc2rR/+0LY1AmjW2HG/Nq0K0mS1GCq1YO4JCL2Av4eWAHcC/xzldqSVLTbFz6fHPbY3F0qr7ae5HTTOiCfT07twZQkSRq0Ie9BjIgXAb/JzCeB7wEHDXUbkurMps7BlQ+l7SWn9iJKkiQNypD3IGbmc8BFQ31dSXVsdOvgyodSkcmpJElSg6nWENNvRMTfRMT4iNi756tKbUkq2oz50NS8bVlTc6m82opMTiVJkhpMtRLEPwMupDTEdFX5q30gL4yIUyJibUQ8EBHz+qkzJyLujYh7IuILQxa1pJ0zbQ7MWgyjxwNRepy1uDZDPItMTiVJkhpMVVYxzcyJO/O6iBgFXA2cBHQCd0XEisy8t6LOJOAS4JjMfDIiXjEUMUtDbfnqLhatXMv6jd2MbWlm7szJzJ4+ruiwqmfanGLm/PW06SqmkiRJu6wqCWJEnNtXeWZev4OXHgk8kJkPla9zI3AGpVVQe7wfuLq8CA6Z+etdj1gaWstXd3HJsjV0b94CQNfGbi5ZtgagsZPEohSVnEqSJDWYag0xPaLi60+ABcDpA3jdOGBdxXFnuazSHwF/FBHfj4gfRcQpux6uNLQWrVy7NTns0b15C4tWri0oIkmSJGnHqjXE9M8rjyNiNPDZAbw0+rpcr+PdgEnACUAr8N8RMSUzN/Zq83zgfIADDjhgYIFLQ2T9xu5BlUuSJEn1oFo9iL39nlJStyOdwPiK41ZgfR91vpyZmzPzYWBtX9fOzCWZ2ZaZbWPGjNnJsKWdM7aleVDlkiRJUj2oSoIYEbdExIry11coJXFfHsBL7wImRcTEiNgdOBtY0avOcuAN5Xb2pTTk9KGhi17adXNnTqa5adQ2Zc1No5g7c3JBEUmSJEk7VpUhpsAVFc+fBX6RmTvctTozn42Ii4CVwCjgmsy8JyIWAu2ZuaJ87uSIuBfYAszNzMeH/i1IO69nIZoRtYqpJEmShr3I7D3FbwguGjEReDQzny4fNwP7ZeYjQ97YALS1tWV7+4C2YZQkSZKkhhMRqzKzbUf1qjUH8b+A5yqOt5TLJEmSJEl1qloJ4m6Z+Yeeg/Lz3avUliRJkiRpCFQrQdwQEVv3PYyIM4DHqtSWJEmSJGkIVGuRmguAz0fEVeXjTuDcKrUlqVLHUrh9IWzqhNGtMGM+TJtTdFSSJEkaBqqSIGbmg8DrIuJllBbCeaoa7UjqpWMp3HIxbO4uHW9aVzoGk0RJkiTtULX2QfyniGjJzN9m5lMRsVdE/GM12pJU4faFzyeHPTZ3l8olSZKkHajWHMRTM3Njz0FmPgm8qUptSeqxqZ/tRvsrlyRJkipUK0EcFREv7jko74P44u3UlzQURrcOrlySJEmqUK0E8XPA7RHx3oh4L/AN4LoqtSWpx4z50NS8bVlTc6lckiRJ2oFqLVJzeUR0AG8EAvgacGA12pJUoWchGlcxlSRJ0k6o1jYXAL8CngPmAA8DX6xiW5J6TJtjQihJkqSdMqQJYkT8EXA2cA7wOHATpW0u3jCU7UgDtXx1F4tWrmX9xm7GtjQzd+ZkZk8fV3RYkiRJUl0a6h7EnwH/DczKzAcAIuIvh7gNaUCWr+7ikmVr6N68BYCujd1csmwNgEmiJEmS1IehXqTmLZSGln47Ij4ZETMozUGUam7RyrVbk8Me3Zu3sGjl2toE0LEUrpwCC1pKjx1La9OuJEmStJOGNEHMzC9l5tuBQ4DvAH8J7BcR/xERJw9lW9KOrN/YPajyIdWxFG65GDatA7L0eMvFJomSJEmqa1XZ5iIzf5eZn8/M04BW4G5gXjXakvoztqV5UOVD6vaFsLlXIrq5u1QuSZIk1alq7YO4VWY+kZn/mZknVrstqdLcmZNpbhq1TVlz0yjmzpxc/cY3dQ6uXJIkSaoD1dzmQipUz0I0haxiOrq1PLy0j3JJkiSpTpkgqqHNnj6umBVLZ8wvzTmsHGba1FwqlyRJkupU1YeYSiPStDkwazGMHg9E6XHWYjewlyRJUl2rux7EiDgF+FdgFPCpzLys1/nzgEVAV7noqsz8VE2D1PDRsbS0MMymztLwzhnza5ekTZtjQihJkqRhpa4SxIgYBVwNnAR0AndFxIrMvLdX1Zsy86KaB6jhpWeriZ5hnj1bTYCJmyRJktSHehtieiTwQGY+lJl/AG4Ezig4Jg1XbjUhSZIkDUq9JYjjgMqlHzvLZb29JSI6IuLmiBhfm9A07LjVhCRJkjQo9ZYgRh9l2ev4FmBCZk4Dvglc1+eFIs6PiPaIaN+wYcMQh6lhob8tJdxqQpIkSepTvSWInUBlj2ArsL6yQmY+npnPlA8/CfxxXxfKzCWZ2ZaZbWPGjKlKsKpzM+aXtpao5FYTkiRJUr/qapEa4C5gUkRMpLRK6dnAOyorRMT+mflo+fB04L7ahqidsXx1V+03rO9ZiKaoVUwlSZKkYaauEsTMfDYiLgJWUtrm4prMvCciFgLtmbkCuDgiTgeeBZ4AzissYA3I8tVdXLJsDd2btwDQtbGbS5atAahNkmhCKEmSJA1IZPae4td42trasr29vegwRqxjLvsWXRu7X1A+rqWZ7887sYCIJEmSpJElIlZlZtuO6tXbHEQ1oPV9JIfbK5ckSZJUDBNEVd3YluZBlUuSJEkqhgmiqm7uzMk0N43apqy5aRRzZ04uKCJJkiRJfamrRWrUmHoWoqn5KqaSJEmSBsUEUTUxe/o4E0JJkiSpzjnEVJIkSZIEmCBKkiRJkspMECVJkiRJgAmiJEmSJKnMBFGSJEmSBJggSpIkSZLKTBAlSZIkSYAJoiRJkiSpzARRkiRJkgSYIEqSJEmSykwQJUmSJEkA7FZ0AKqd5au7WLRyLes3djO2pZm5Mycze/q4osOSJEmSVCdMEEeI5au7uGTZGro3bwGga2M3lyxbA2CSKEmSJAlwiOmIsWjl2q3JYY/uzVtYtHJtQRFJkiRJqjd1lyBGxCkRsTYiHoiIedup99aIyIhoq2V8w9X6jd2DKpckSZI08tRVghgRo4CrgVOBQ4FzIuLQPurtCVwM3FnbCIevsS3Ngyofch1L4copsKCl9NixtDbtSpIkSRqwukoQgSOBBzLzocz8A3AjcEYf9f4fcDnwdC2DG87mzpxMc9Oobcqam0Yxd+bk6jfesRRuuRg2rQOy9HjLxSaJkiRJUp2ptwRxHLCu4rizXLZVREwHxmfmV2oZ2HA3e/o4Lj1rKuNamglgXEszl541tTYL1Ny+EDb3Gsq6ubtULkmSJKlu1NsqptFHWW49GfEi4ErgvB1eKOJ84HyAAw44YIjCG95mTx9XzIqlmzoHVy5JkiSpEPXWg9gJjK84bgXWVxzvCUwBvhMRjwCvA1b0tVBNZi7JzLbMbBszZkwVQ9YOjW4dXLkkSZKkQtRbgngXMCkiJkbE7sDZwIqek5m5KTP3zcwJmTkB+BFwema2FxOuBmTGfGjqtRhOU3OpXJIkSVLdqKsEMTOfBS4CVgL3AUsz856IWBgRpxcbnXbatDkwazGMHg9E6XHW4lK5JEmSpLoRmbnjWsNcW1tbtrfbyUjH0tLCMJs6S8M7Z8w3SZMkSZJGgIhYlZk73EO+3hapUbX0bDXRs5poz1YTYJIoSZIkCaizIaaqIreakCRJkrQDJogjhVtNSJIkSdoBE8SRwq0mJEmSJO2ACeJI4VYTkiRJknbABHGkcKsJSZIkSTvgKqYjybQ5JoSSJEkBUaY3AAAgAElEQVSS+mUPoiRJkiQJMEGUJEmSJJWZIEqSJEmSABNESZIkSVKZCaIkSZIkCTBBlCRJkiSVmSBKkiRJkgATREmSJElSmQmiJEmSJAkwQZQkSZIklZkgSpIkSZIAE0RJkiRJUpkJoiRJkiQJqMMEMSJOiYi1EfFARMzr4/wFEbEmIu6OiDsi4tAi4pQkSZKkRlNXCWJEjAKuBk4FDgXO6SMB/EJmTs3Mw4HLgY/VOExJkiRJakh1lSACRwIPZOZDmfkH4EbgjMoKmfmbisOXAlnD+CRJkiSpYe1WdAC9jAPWVRx3Akf1rhQRFwJ/BewOnFib0CRJkiSpsdVbD2L0UfaCHsLMvDozXwV8GPj7Pi8UcX5EtEdE+4YNG4Y4TEmSJElqPPWWIHYC4yuOW4H126l/IzC7rxOZuSQz2zKzbcyYMUMYoiRJkiQ1pnpLEO8CJkXExIjYHTgbWFFZISImVRy+Gbi/hvFJkiRJUsOqqzmImflsRFwErARGAddk5j0RsRBoz8wVwEUR8UZgM/Ak8KfFRSxJkiRJjaOuEkSAzLwVuLVX2fyK539R86AkSZIkaQSotyGmkiRJkqSCmCBKkiRJkgATREmSJElSWd3NQZQkSdLIsnnzZjo7O3n66aeLDkUa9vbYYw9aW1tpamraqdebIBZg+eouFq1cy/qN3YxtaWbuzMnMnj6u6LAkSZIK0dnZyZ577smECROIiKLDkYatzOTxxx+ns7OTiRMn7tQ1HGJaY8tXd3HJsjV0bewmga6N3VyybA3LV3cVHZokSVIhnn76afbZZx+TQ2kXRQT77LPPLvXGmyDW2KKVa+nevGWbsu7NW1i0cm1BEUmSJBXP5FAaGrv6b8kEscbWb+weVLkkSZKq78/+7M94xStewZQpU/qtk5lcfPHFHHzwwUybNo3/+Z//2XruuuuuY9KkSUyaNInrrrtuh+195zvf4bTTThuS2HfGtddey/r16wtrf6h97WtfY/LkyRx88MFcdtllfdb52Mc+xqGHHsq0adOYMWMGv/jFL7aeGzVqFIcffjiHH344p59++g7be+SRR7Z7r1Tb8uXLuffee6tybRPEGhvb0jyockmSJFXfeeedx9e+9rXt1rntttu4//77uf/++1myZAkf/OAHAXjiiSf4yEc+wp133smPf/xjPvKRj/Dkk0/uVBzPPvvsTr1usBopQdyyZQsXXnght912G/feey833HBDn8nT9OnTaW9vp6Ojg7e+9a387d/+7dZzzc3N3H333dx9992sWLFip2Op1c/PBLGBzJ05meamUduUNTeNYu7MyQVFJEmSNLwsX93FMZd9i4nzvsoxl31rSNZyOO6449h77723W+fLX/4y5557LhHB6173OjZu3Mijjz7KypUrOemkk9h7773Za6+9OOmkk/pMNr/2ta9xyCGHcOyxx7Js2bKt5QsWLOD888/n5JNP5txzz+Xpp5/mPe95D1OnTmX69Ol8+9vfBkpJ3RlnnMEpp5zC5MmT+chHPrL1Gh/72MeYMmUKU6ZM4eMf/zjwwl6uK664ggULFnDzzTfT3t7OO9/5Tg4//HC6u2s8kq1jKVw5BRa0lB47lu7S5X784x9z8MEHc9BBB7H77rtz9tln8+Uvf/kF9d7whjfwkpe8BIDXve51dHZ2DqqdVatWcdhhh3H00Udz9dVXby2/9tpredvb3sasWbM4+eSTyUzmzp3LlClTmDp1KjfddBNQ6jU+7rjjOPPMMzn00EO54IILeO655wC44YYbmDp1KlOmTOHDH/7w1mu/7GUv2/r85ptv5rzzzuMHP/gBK1asYO7cuRx++OE8+OCDg3ofO+IqpjXWs1qpq5hKkiQNXs+Cfz1rOvQs+AdU/feprq4uxo8fv/W4tbWVrq6ufssrPf3007z//e/nW9/6FgcffDBvf/vbtzm/atUq7rjjDpqbm/mXf/kXANasWcPPfvYzTj75ZH7+858DpWTopz/9KS95yUs44ogjePOb30xE8JnPfIY777yTzOSoo47i+OOPZ6+99urzfbz1rW/lqquu4oorrqCtrW1IvjcD1rEUbrkYNpeT0k3rSscA0+bs1CX7+v7feeed233Npz/9aU499dStx08//TRtbW3stttuzJs3j9mzZ7/gNe95z3v4t3/7N44//njmzp27zbkf/vCHdHR0sPfee/PFL36Ru+++m5/85Cc89thjHHHEERx33HFA6ed37733cuCBB3LKKaewbNkyXv/61/PhD3+YVatWsddee3HyySezfPnyPmMAeP3rX8/pp5/Oaaedxlvf+tYBf58GygSxALOnjzMhlCRJ2gnbW/Cv2r9fZeYLyiKi3/JKP/vZz5g4cSKTJk0C4F3vehdLlizZev7000+nubk05eiOO+7gz//8zwE45JBDOPDAA7cmiCeddBL77LMPAGeddRZ33HEHEcGZZ57JS1/60q3l//3f/z2guXQ1d/vC55PDHpu7S+U7mSAO5Ptf6XOf+xzt7e1897vf3Vr2y1/+krFjx/LQQw9x4oknMnXqVF71qldtPb9p0yY2btzI8ccfD8C73/1ubrvttq3ne3qQofTzO+eccxg1ahT77bcfxx9/PHfddRcvf/nLOfLIIznooIMAOOecc7jjjjtoamrihBNOYMyYMQC8853v5Hvf+16/CWK1OcS0CEPcrS5JkjRSFLngX2trK+vWrdt63NnZydixY/st7217SUtPcgd9Jzz9XaO/BBVgt9122zqEEdilrQ+GzKZ+hnX2Vz4AA/3+A3zzm9/kox/9KCtWrODFL37x1vKe+gcddBAnnHACq1ev3uZ1mVnzn1/v+rX6+Zkg1lpPt/qmdUA+361ukihJkrRDRS74d/rpp3P99deTmfzoRz9i9OjR7L///sycOZOvf/3rPPnkkzz55JN8/etfZ+bMmdu89pBDDuHhhx/eOl/shhtu6Led4447js9//vMA/PznP+eXv/wlkyeX1qv4xje+wRNPPEF3dzfLly/nmGOO4bjjjmP58uX8/ve/53e/+x1f+tKX+JM/+RP2228/fv3rX/P444/zzDPP8JWvfGVrG3vuuSdPPfXUUH+Ldmx06+DKB+CII47g/vvv5+GHH+YPf/gDN954Y5+9p6tXr+YDH/gAK1as4BWveMXW8ieffJJnnnkGgMcee4zvf//7HHroodu8tqWlhdGjR3PHHXcAbP359OW4447jpptuYsuWLWzYsIHvfe97HHnkkUBpiOnDDz/Mc889x0033cSxxx7LUUcdxXe/+10ee+wxtmzZwg033LC1p3K//fbjvvvu47nnnuNLX/rS1jaq+fMzQay17XWrS5IkabuqteDfOeecw9FHH83atWtpbW3l05/+NACf+MQn+MQnPgHAm970Jg466CAOPvhg3v/+9/Pv//7vAOy99978wz/8A0cccQRHHHEE8+fPf8GCN3vssQdLlizhzW9+M8ceeywHHnhgv7F86EMfYsuWLUydOpW3v/3tXHvttVt7u4499lje/e53c/jhh/OWt7yFtrY2Xvva13Leeedx5JFHctRRR/G+972P6dOn09TUxPz58znqqKM47bTTOOSQQ7a2cd5553HBBRfUfpGaGfOhqVcy39RcKt9Ju+22G1dddRUzZ87k1a9+NXPmzOE1r3kNAPPnz9+6KuncuXP57W9/y9ve9rZttrO47777aGtr47DDDuMNb3gD8+bNe0GCCPCZz3yGCy+8kKOPPnrrcOC+nHnmmUybNo3DDjuME088kcsvv5xXvvKVABx99NHMmzePKVOmMHHiRM4880z2339/Lr30Ut7whjdw2GGH8drXvpYzzjgDgMsuu4zTTjuNE088kf33339rG2effTaLFi1i+vTpQ75ITWyvS7NRtLW1ZXt7e9FhlCxoAfr6ngcs2FjraCRJkgp333338epXv3rA9Zev7hqRC/5de+21tLe3c9VVVxUdyq7pWFrqHNnUWeo5nDF/p+cfDiff+c53uOKKK7bpya2Wvv5NRcSqzNzhqkQuUlNro1vLw0v7KJckSdIOueDfMDdtzohICIcrh5jWWhW61SVJktT4zjvvvOHfeziCnXDCCTXpPdxVJoi1Nm0OzFoMo8cDUXqctdi/okiSJEkqnENMi2C3uiRJ0jZ2tI2ApIHZ1TVm6q4HMSJOiYi1EfFARMzr4/xfRcS9EdEREbdHRP9LQEmSJKnu7bHHHjz++OO7/IutNNJlJo8//jh77LHHTl+jrnoQI2IUcDVwEtAJ3BURKzLz3opqq4G2zPx9RHwQuBx4e+2jlSRJ0lBobW2ls7OTDRs2FB2KNOztsccetLbu/AKYdZUgAkcCD2TmQwARcSNwBrA1QczMb1fU/xHwrppGKEmSpCHV1NTExIkTiw5DEvU3xHQcULkHRGe5rD/vBW6rakSSJEmSNELUWw9iXzOT+xyMHhHvAtqA4/s5fz5wPsABBxwwVPFJkiRJUsOqtx7ETmB8xXErsL53pYh4I/B/gNMz85m+LpSZSzKzLTPbxowZU5VgJUmSJKmRRD2tFhURuwE/B2YAXcBdwDsy856KOtOBm4FTMvP+AV53A/CLoY94l+0LPFZ0EGpY3l+qJu8vVZP3l6rNe0zVVK/314GZucOes7pKEAEi4k3Ax4FRwDWZ+dGIWAi0Z+aKiPgmMBV4tPySX2bm6QWFu0sioj0z24qOQ43J+0vV5P2lavL+UrV5j6mahvv9VW9zEMnMW4Fbe5XNr3j+xpoHJUmSJEkjQL3NQZQkSZIkFcQEsVhLig5ADc37S9Xk/aVq8v5StXmPqZqG9f1Vd3MQJUmSJEnFsAdRkiRJkgSYIBYiIk6JiLUR8UBEzCs6HjWeiHgkItZExN0R0V50PBreIuKaiPh1RPy0omzviPhGRNxfftyryBg1fPVzfy2IiK7yZ9jd5RXOpUGLiPER8e2IuC8i7omIvyiX+xmmXbad+2tYf4Y5xLTGImIUpb0eTwI6Ke31eE5m3ltoYGooEfEI0JaZ9bgHj4aZiDgO+C1wfWZOKZddDjyRmZeV/9C1V2Z+uMg4NTz1c38tAH6bmVcUGZuGv4jYH9g/M/8nIvYEVgGzgfPwM0y7aDv31xyG8WeYPYi1dyTwQGY+lJl/AG4Ezig4JknqV2Z+D3iiV/EZwHXl59dR+g9RGrR+7i9pSGTmo5n5P+XnTwH3AePwM0xDYDv317Bmglh744B1FcedNMCNpLqTwNcjYlVEnF90MGpI+2Xmo1D6DxJ4RcHxqPFcFBEd5SGoDv/TLouICcB04E78DNMQ63V/wTD+DDNBrL3oo8xxvhpqx2Tma4FTgQvLQ7gkabj4D+BVwOHAo8C/FBuOhruIeBnwReD/y8zfFB2PGksf99ew/gwzQay9TmB8xXErsL6gWNSgMnN9+fHXwJcoDW2WhtL/lude9MzB+HXB8aiBZOb/ZuaWzHwO+CR+hmkXREQTpV/eP5+Zy8rFfoZpSPR1fw33zzATxNq7C5gUERMjYnfgbGBFwTGpgUTES8sTpYmIlwInAz/d/qukQVsB/Gn5+Z8CXy4wFjWYnl/cy87EzzDtpIgI4NPAfZn5sYpTfoZpl/V3fw33zzBXMS1AeanbjwOjgGsy86MFh6QGEhEHUeo1BNgN+IL3mHZFRNwAnADsC/wv8H+B5cBS4ADgl8DbMtOFRjRo/dxfJ1AampXAI8AHeuaLSYMREccC/w2sAZ4rF/8dpXlifoZpl2zn/jqHYfwZZoIoSZIkSQIcYipJkiRJKjNBlCRJkiQBJoiSJEmSpDITREmSJEkSYIIoSZIkSSozQZQkaYAiYktE3F3xNW8Irz0hIobVXlmSpMazW9EBSJI0jHRn5uFFByFJUrXYgyhJ0i6KiEci4p8j4sflr4PL5QdGxO0R0VF+PKBcvl9EfCkiflL+en35UqMi4pMRcU9EfD0imgt7U5KkEckEUZKkgWvuNcT07RXnfpOZRwJXAR8vl10FXJ+Z04DPA4vL5YuB72bmYcBrgXvK5ZOAqzPzNcBG4C1Vfj+SJG0jMrPoGCRJGhYi4reZ+bI+yh8BTszMhyKiCfhVZu4TEY8B+2fm5nL5o5m5b0RsAFoz85mKa0wAvpGZk8rHHwaaMvMfq//OJEkqsQdRkqShkf08769OX56peL4F1wqQJNWYCaIkSUPj7RWPPyw//wFwdvn5O4E7ys9vBz4IEBGjIuLltQpSkqTt8S+TkiQNXHNE3F1x/LXM7Nnq4sURcSelP76eUy67GLgmIuYCG4D3lMv/AlgSEe+l1FP4QeDRqkcvSdIOOAdRkqRdVJ6D2JaZjxUdiyRJu8IhppIkSZIkwB5ESZIkSVKZPYiSJJVFxISIyIhwjr4kaUQyQZQkNYyIWBkRC/soPyMifmXiJ0nS9pkgSpIaybXAuyMiepW/G/h8Zj5b+5CGRpT4/7Ykqar8j0aS1EiWA3sDf9JTEBF7AacB15eP3xwRqyPiNxGxLiIWDPTiETEvIh6MiKci4t6IOLPX+fdHxH0V519bLh8fEcsiYkNEPB4RV5XLF0TE5ypev80Q14j4TkR8NCK+D/weOCgi3lPRxkMR8YFeMZwREXeX39+DEXFKRLwtIlb1qvfXEbF8oO9dkjQymCBKkhpGZnYDS4FzK4rnAD/LzJ+Uj39XPt8CvBn4YETMHmATD1JKPkcDHwE+FxH7A0TE24AF5Wu/HDgdeDwiRgFfAX4BTADGATcO4m29Gzgf2LN8jV9TSnhfTmlfxSsrEtEjKSXCc8vv7zjgEWAFMDEiXl1x3XcBnx1EHJKkEcAEUZLUaK4D3hYRzeXjc8tlAGTmdzJzTWY+l5kdwA3A8QO5cGb+V2auL7/2JuB+4Mjy6fcBl2fmXVnyQGb+onx+LDA3M3+XmU9n5h2DeD/XZuY9mflsZm7OzK9m5oPlNr4LfJ3ne0zfC1yTmd8ox9iVmT/LzGeAmyglhUTEayglq18ZRBySpBHABFGS1FDKydcG4IyIOAg4AvhCz/mIOCoivl0e7rkJuADYdyDXjohzy8M3N0bERmBKxWvHU+ph7G088ItdmP+4rlcMp0bEjyLiiXIMbxpADFBKkt9Rnp/5bmBpOXGUJGkrE0RJUiO6nlLP4buBr2fm/1ac+wKlIZfjM3M08Amg96I2LxARBwKfBC4C9snMFuCnFa9dB7yqj5euAw7oZwXV3wEvqTh+ZR91tm5YHBEvBr4IXAHsV47h1gHEQGb+CPgDpd7Gd+DwUklSH0wQJUmN6HrgjcD7qRheWrYn8ERmPl2es/eOAV7zpZSStQ0AEfEeSj2IPT4F/E1E/HF5xdGDy0nlj4FHgcsi4qURsUdEHFN+zd3AcRFxQESMBi7ZQQy7Ay8ux/BsRJwKnFxx/tPAeyJiRkS8KCLGRcQhFeevB64Cnh3kMFdJ0ghhgihJajiZ+QjwA0pJ3Ypepz8ELIyIp4D5lBa1Gcg17wX+Bfgh8L/AVOD7Fef/C/gopR7KpyivqJqZW4BZwMHAL4FO4O3l13yD0tzADmAVO5gTmJlPAReXY36SUnK7ouL8jykvXANsAr4LHFhxic9SSmrtPZQk9Skyc8e1JEnSsFdeuOfXwGsz8/6i45Ek1R97ECVJGjk+CNxlcihJ6k9fE+YlSVKDiYhHKC1mM9A9HyVJI5BDTCVJkiRJgENMJUmSJEllVU0QI+KUiFgbEQ9ExLw+zl8QEWvKmw7fERGHlssnRER3ufzuiPhExWv+uPyaByJicXnDX0mSJEnSLqraENOIGAX8HDiJ0pLedwHnlJcJ76nz8sz8Tfn56cCHMvOUiJgAfCUzp/Rx3R8DfwH8iNLmwIsz87btxbLvvvvmhAkThuJtSZIkSdKws2rVqscyc8yO6lVzkZojgQcy8yGAiLgROAPYmiD2JIdlPRsQ9ysi9gdenpk/LB9fT2my/XYTxAkTJtDe3r4z70GSJEmShr2I+MVA6lVziOk4YF3FcWe5bBsRcWFEPAhcTmnz3x4TI2J1RHw3Iv6k4pqdO7qmJEmSJGnwqpkg9jU38AU9hJl5dWa+Cvgw8Pfl4keBAzJzOvBXwBci4uUDvSZARJwfEe0R0b5hw4adegOSJEmSNJJUM0HsBMZXHLcC67dT/0bKezNl5jOZ+Xj5+SrgQeCPytdsHcg1M3NJZrZlZtuYMTscaitJkiRJI141E8S7gEkRMTEidgfOBlZUVoiISRWHbwbuL5ePKS9yQ0QcBEwCHsrMR4GnIuJ15dVLzwW+XMX3IEmSJEkjRtUWqcnMZyPiImAlMAq4JjPviYiFQHtmrgAuiog3ApuBJ4E/Lb/8OGBhRDwLbAEuyMwnyuc+CFwLNFNanGa7C9RIkiRJkgamattc1JO2trZ0FVNJkiRJI1VErMrMth3Vq+YQU0mSNBx1LIUrp8CCltJjx9KiI5Ik1Ug190GUJEnDTcdSuOVi2NxdOt60rnQMMG1OcXFJkmrCHkRJkvS82xc+nxz22NxdKpckNTwTREmS9LxNnYMrlyQ1FBNESZL0vNGtgyuXJDUUE0RJkvS8GfOhqXnbsqbmUrkkqeGZIEqSpOdNmwOzFsPo8UCUHmctdoEaSRohXMVUkiRta9ocE0JJGqHsQZQkSZIkASaIkiRJkqQyE0RJkiRJEmCCKEmSJEkqM0GUJEn1oWMpXDkFFrSUHjuWFh2RJI04rmIqSZKK17EUbrkYNneXjjetKx2DK6pKUg3ZgyhJkop3+8Lnk8Mem7tL5ZKkmjFBlCRJxdvUObhySVJVmCBKkqTijW4dXLkkqSpMECVJUvFmzIem5m3LmppL5ZKkmjFBlCRJxZs2B2YthtHjgSg9zlrsAjWSVGOuYipJkurDtDn/f3t3H2d1Xef///EKBx2zBTVSYVBx5YcRILiDZrp4lWKliGaoW5nVZrb6dbduXxK/Wy7623664la/0q3cLqzd8iJTom2VTO2CLowhCK8iSUsHbAUVu3BUxNf3j3OGDjgzzDDzmQ9zzuN+u83tnM/7fC7en5kPh/M8n/eFgVDS0LXypsrAWs+0V5rHH3fJkHxPMyBKkiRJUn/U0VQ9NjGVJEmSpP6oo6l6DIiSJEmS1B91NFWPAVGSJEmS+qOOpuopNCBGxIkRsSoiVkfEvC5ePy8i7o2IFRGxJCImVsuPj4hl1deWRcSxNdt8r7rPFdWf1xR5DpIkSZLUozqaqqewQWoiYhhwDXA80A4sjYhFmflAzWpfy8zPVtefBXwcOBFYD5ycmWsjYhKwGBhTs93bM7OtqLpLkiRJUq91DkTjKKY9OhRYnZkPA0TEDcApwOaAmJm/r1n/lUBWy5fXlN8P7BIRO2fm8wXWV5IkSZK2T51M1VNkQBwDPFaz3A4ctvVKEXE+8CFgOHDs1q8DbwWWbxUOvxQRm4BvAP+cmTlgtZYkSZKkBlVkH8ToouxlQS4zr8nMvwQuAj6yxQ4iXgf8C/D+muK3Z+Zk4K+rP+/s8uAR50ZEW0S0rVu3bjtPQZIkSZIaR5EBsR0YW7PcAqztYf0bgNmdCxHRAtwKnJ2Zv+4sz8w11cc/AF+j0pT1ZTLz2sxszczWUaNGbfdJSJIkSVKjKDIgLgXGR8S4iBgOnAksql0hIsbXLL4FeKhaPhL4NnBxZv6oZv2dIuLV1edNwEnAfQWegyRJkiQ1jML6IGbmixFxAZURSIcBX8zM+yPiMqAtMxcBF0TEG4GNwNPAu6qbXwAcCHw0Ij5aLTsB+BOwuBoOhwHfBf69qHOQJEmqWytvqosRFyUNrGiE8V1aW1uzrc1ZMSRJkoBKOPzWhbCx489lTc1w8qcMiVKdiohlmdm6rfWKbGIqSZKkHdGdl20ZDqGyfOdl5dRH0g7DgChJktRonmnvW7mkhlHkPIiSJEnaEY1ogWce67q8aPZ9HDQLl69hweJVrN3QweiRzcydOYHZ08aUXS3t4LyDKEmS1GiOu6TS57BWU3OlvEidfR+feQzIyuO3LqyUa0AtXL6Gi2+5lzUbOkhgzYYOLr7lXhYuX1N21bSDMyBKkiQ1milzKgPSjBgLROVxMAaose/joFmweBUdGzdtUdaxcRMLFq8qqUYaKmxiKkmS1IimzBn8pp32fRw0azd09Klc6uQdREmSJA2O7vo4DkbfR6g0Zf3EJJg/svJYx01bR49s7lO51MmAKEnbq4E+aEjSgCir7yM0XP/HuTMn0Nw0bIuy5qZhzJ05oaQaaagwIErS9miwDxqSNCDK6vsIDdf/cfa0MVx+2mTGjGwmgDEjm7n8tMmOYqptiswsuw6Fa21tzba2trKrIamefGJSN0PEj4UP3jf49ZEk9Wz+SKCrz70B8zcMdm2kQRcRyzKzdVvreQdRkraHAy1I0tBSdv9HaYgwIErS9vCDhiQNLSX2f1y4fA1HXHEX4+Z9myOuuMu5CIvk+AD9ZkCUpO1R5kALkqS+K6n/oxPWDyLHBxgQzoMoSduj8wPFnZdVmpWOaKmEw8GeU0yS1HslzP24YPEqjt/0fT48/CZGx3rW5qu58sU5LFg8vK4HjFm4fA0LFq9i7YYORo9sZu7MCcWfb08DEfn/c68ZECVpe5UxybQkaUhp/f0dXN70eXaNFwBoifVc0fR5Lv49wLGl1q0onXdNOzZuAv581xQoNiQ6PsCAsImpJEmSGkIZfQEvHv71zeGw067xAhcP/3rhxy7LgsWrNofDTh0bN7Fg8apiD+z4AAPCgChJ0g5q6aLP8bv5B/LSP43gd/MPZOmiz5VdJWnIKqsv4F6s71N5PVi7oaNP5QPG8QEGhAFRkqQd0NJFn2PSso+wN+t4RcDerGPSso8YEqXtVNZdrejm7lV35fVg9MjmPpUPmJIGIqo3BkRJUu+VOXx4gw1dPvbnC2jeqllac7zA2J8vKKlGKoLTHwwe72oNnrkzJ9DcNGyLsuamYcydOaH4g0+ZAx+8D+ZvqIAt3d8AACAASURBVDwaDvvMQWokSb3TOXx45whxncOHQ/H/AZd57JK8JtdBdFVefLO0UkYfbEClDeTRoEaPbGZNF2FwUO5qQUONet15/fo+MjRFZpZdh8K1trZmW1tb2dWQpKHtE5Oqc0ttZcTYyre0dXrsssLS7+YfyN6se3k5o9h7/urCjrt1aIHKN/+XnzbZD3cD7Igr7uoysIwZ2cyP5tXn6JZl8tpWo4uIZZnZuq31bGIqSeqdMocPL+nYZU5w/dghc+nI4VuUdeRwHjtkbqHHLW30wQZUWpPHBjV72hguP20yY0Y2E1SCuOFQejmbmEqSemdESzd38QZhoIWSjt1TWCr6Q+X0We9nKZW+iK/J9TwRr+axv5rL9FnvL/S4hpbBU1qTxwY2e9oYA2Gds4l8/xV6BzEiToyIVRGxOiLmdfH6eRFxb0SsiIglETGx5rWLq9utioiZvd2nJKkgZQ60UNKxyw5L02e9n73nr+YVl25g7/mrCw+HUOLogyUrY7CYUgfyKJED86goZbb66Dx+PVzbhQXEiBgGXAO8CZgInFUbAKu+lpmTM3MqcCXw8eq2E4EzgdcBJwL/FhHDerlPSVIRyhw+vKRjN2JYasTQUtaHykZs8lj2B3jVtzKbyNfTtV1kE9NDgdWZ+TBARNwAnAI80LlCZv6+Zv1XAp0j5pwC3JCZzwOPRMTq6v7Y1j4lSQWaMqe8kfdKOPbcmRO6HNSinsNSI44+WGZT4kZr8ljm71r1r8xWH/V0bRcZEMcAtR1G2oHDtl4pIs4HPgQMBzqH7BoD/HSrbTt/s9vcp9TIbHsvDZxGDEvQeKFl7YYOZr1iCR/e6SZGx3rW5qu58sU5fGvDkWVXrVBl/H9RdrNt1bcy+/XW07VdZEDsYvYmXjanRmZeA1wTEX8DfAR4Vw/bdtUktst5OiLiXOBcgH333beXVZYG0MqbBn3OI+fUkgZeo4WlRvSu3X7Ghzd+nl3jBQBaYj1XNH2ePZqGA28pt3IFKev/CwfmUZHKbPVRT9d2kYPUtANja5ZbgLU9rH8DMHsb2/Z6n5l5bWa2ZmbrqFGj+lh1qZ86J/V+5jEg/zyp98qbCj1sww5Pv/Kmyjx580dWHgv+PUuqLx9uunFzOOy0a7zAh5tuLKlGxSvr/4tG7OOqwVNmv956uraLvIO4FBgfEeOANVQGnfmb2hUiYnxmPlRdfAvQ+XwR8LWI+DgwGhgP/IzKncUe9yntEO68DDZu9S3Sxo5KeYF3EeupeUOvdYbxzt93ZxiH8vrKSRpSdu34XZ/K60FZ/180arNtDZ6yWn3U07VdWEDMzBcj4gJgMTAM+GJm3h8RlwFtmbkIuCAi3ghsBJ6m0ryU6no3URl85kXg/MzcBNDVPos6B2m7lTSp9+iRzfzV7+94WT+aZX9xfKHHLVVJYbx0JTRhlupWmXN8lqTM5nA221a9qpdru8g7iGTmfwP/vVXZJTXP/76HbT8GfKw3+5R2OCV92PjkxIeYtOzzNNf0o/mXps9z38T9+fMYUHWmpDBeKu+aSgPruEu2/DcFgzfHZ0kacYTehlTml4l+kTlkFdkHUWpcJU3qPf3Xn94cDjs1xwtM//WnCz1uqboL3XX8zX+Pd00l9V2Zc3yW1Ie6EedgbDgljYdQ+rHVb4XeQZQaVueHisH+5qwR76Y14Df/Dfl3lopWxhyfJbcGqJfmcOpGmV0wGrX7R50wIEpFKePDRgP2oyktjJepEf/OUj3yQ7SKVOaXiX6ROaTZxFSqJyU1bS3dlDnwwftg/obKY71/sGrUv7NUb/wQrSKV2QWjEbt/1BEDolRPGrAfTUMq8+8saeD4IVpFKvPLRL/IHNIiM8uuQ+FaW1uzra2t7GpI9WvrfjRQ+Y/A0KJ64Eh8KorvnSqao5iqRkQsy8zWba5nQJTUb5+Y1E2fuLGVJp/SUOUHeBXND9GSBklvA6KD1EjqP/vRqF45iIiKVsaAZpLUA/sgSuo/+9GoXvnlhySpwRgQJfWfndFVr/zyQ5LUYAyIkvrPUTVVr/zyQ5LUYOyDKGlg2I9G9ajzmnYQEUlSgzAgSpLUE7/8kCQ1EJuYSpIkSZIAA6IkSZIkqcqAKEmSJEkC7IMoaYAsXL6GBYtXsXZDB6NHNjN35gRmTxtTdrUkSZLUBwZESf22cPkaLr7lXjo2bgJgzYYOLr7lXgBDoiRJ0hBiE1NJ/bZg8arN4bBTx8ZNLFi8qqQaSZIkaXsYECX129oNHX0qlyRJ0o7JgCip30aPbO5TuSRJknZMBkRJ/TZ35gSam4ZtUdbcNIy5MyeUVCNJQ9HC5Ws44oq7GDfv2xxxxV0sXL6m7CpJUsNxkBrVNUfWHBydv1N/15K2l4NdSdKOwYCouuWHjcE1e9oYf6+StltPg1353iJJg6fQgBgRJwL/PzAM+HxmXrHV6x8C/hZ4EVgHvCczfxsRxwCfqFn1IODMzFwYEdcBRwHPVF87JzNXFHke6r8y7uQtWLyK4zd9nw8Pv4nRsZ61+WqufHEOCxYP98OGNATZIqC+OdiVJO0YCguIETEMuAY4HmgHlkbEosx8oGa15UBrZj4bER8ArgTOyMy7ganV/ewBrAa+U7Pd3My8uai6a2CVdSev9fd3cHnT59k1XgCgJdZzRdPnufj3AMcWdlxpMDRaWLJFQP0bPbKZNV2EQQe7kqTBtc1BaiLigojYfTv2fSiwOjMfzswXgBuAU2pXyMy7M/PZ6uJPgZYu9nM6cFvNehpiypoj7+LhX98cDjvtGi9w8fCvF3pcqWidYWnNhg6SP4eleh7Qw7k265+DXUnSjqE3o5juTeXu300RcWJERC/3PQZ4rGa5vVrWnfcCt3VRfiZw/VZlH4uIlRHxiYjYuZf1UUnKaja0F+v7VC4NFY0Ylmx+WP9mTxvD5adNZszIZgIYM7KZy0+b7B1iSRpk22ximpkfiYiPAicA7waujoibgC9k5q972LSrIJldrhjxDqCVSt/C2vJ9gMnA4prii4HfAcOBa4GLgMu62Oe5wLkA++67bw/VVNHKajYUI1rgmce6LpeGsEYMSzY/bAwOdiVJ5evVPIiZmVRC2e+oDCizO3BzRFzZw2btwNia5RZg7dYrRcQbgX8EZmXm81u9PAe4NTM31tTl8ax4HvgSlaasXdX52sxszczWUaNGbfMcVZzSmg0ddwk0bfXhsam5Ui4NYd2FonoOSzY/lCRpcPSmD+KFEbGMygAyPwImZ+YHgL8C3trDpkuB8RExLiKGU2kqumirfU8DPkclHD7RxT7OYqvmpdW7ilSbus4G7tvWOahcpTUbmjIHTv4UjBgLROXx5E9VygeBEz6rKI0Ylmx+KEnS4IjKzcEeVoi4jEpz0t928dprM/PBHrZ9M/BJKtNcfDEzP1bdX1tmLoqI71JpQvp4dZNHM3NWddv9qQTSsZn5Us0+7wJGUWnCugI4LzP/2NM5tLa2ZltbW4/nKQ2krUdchMoHeD/QaqA02iimkiSpfyJiWWa2bnO9XgTE1wP3Z+YfqsuvAiZm5j0DUtNBYEDUYDviiru67C81ZmQzP5rnFBuSJEkaXL0NiL2ZB/EzwCE1y3/qokxSjUYcRKQReRdPkiTVm94MUhNZc5ux2tyzN8FSaliNOIhIo2nEuQglSVL9601AfLg6UE1T9efvgYeLrpg0lDXiICKNphHnIpQkSfWvNwHxPOANwBoqU1ccRnV+QUldc8TF+mczYkmSVI+22VS0Ov3EmYNQF6muOOFzfXPidkmSVI+2GRAjYhfgvcDrgF06yzPzPQXWS5J2aHNnTuhyKhObEUuSpKGsN01M/wPYG5gJfB9oAf5QZKUkaUdnM2JJklSPejMP4vLMnBYRKzNzSkQ0AYszc8hM5uY8iJIkSZIaWW/nQezNHcSN1ccNETEJGAHs34+6SZIkSZJ2QL2Zz/DaiNgd+AiwCNgN+GihtZIkSZIkDboeA2JEvAL4fWY+DfwAOGBQaiVJ2iEtXL6GBYtXsXZDB6NHNjN35gT7XUqSVEd6bGKamS8BFwxSXSRJO7CFy9dw8S33smZDBwms2dDBxbfcy8Lla8qumiRJGiC96YN4R0T874gYGxF7dP4UXjNJ0g5lweJVW0zrAdCxcRMLFq8qqUaSJGmg9aYPYud8h+fXlCU2N5WkhrJ2Q0efyiVJ0tCzzYCYmeMGoyIqnn2HJPXH6JHNrOkiDI4e2VxCbSRJUhG2GRAj4uyuyjPzKwNfHRWls+9QZ/Owzr5DgCFRUq/MnTlhi/cRgOamYcydOaHEWkmSpIHUmyam02ue7wIcB/wcMCAOIT31HTIgSuqNzvcKWyJIklS/etPE9H/VLkfECOA/CquRCmHfIUkDYfa0MQZCSZLqWG/uIG7tWWD8QFdExbLvkOqZ/WslSZIGRm/6IH6LyqilUJkWYyJwU5GV0sArve/QypvgzsvgmXYY0QLHXQJT5gzOsVXX7F8rSZI0cHpzB/GqmucvAr/NzPaC6qOClNp3aOVN8K0LYWP1DuYzj1WWwZCofrN/rSRJ0sDpTUB8FHg8M58DiIjmiNg/M39TaM004ErrO3TnZX8Oh502dlTKDYjqJ/vXSpIkDZxX9GKdrwMv1SxvqpZJvfNMNzecuyuX+qC7frT2r5UkSeq73gTEnTLzhc6F6vPhxVVJdWdES9/KpT6YO3MCzU3Dtihzbj5JkqTt05uAuC4iZnUuRMQpwPre7DwiToyIVRGxOiLmdfH6hyLigYhYGRF3RsR+Na9tiogV1Z9FNeXjIuKeiHgoIm6MCMPqju64S6Bpq7s5Tc2VcqmfZk8bw+WnTWbMyGYCGDOymctPm2z/Q0mSpO0QmdnzChF/CXwVGF0tagfOzszV29huGPAr4PjqNkuBszLzgZp1jgHuycxnI+IDwNGZeUb1tT9m5m5d7Pcm4JbMvCEiPgv8IjM/01NdWltbs62trcfzVMEcxVSSJEkqTUQsy8zWba23zUFqMvPXwOsjYjcqgfIPvazDocDqzHy4WqEbgFOAzQExM++uWf+nwDt62mFEBHAs8DfVoi8D84EeA6J2AFPmGAglSZKkHdw2m5hGxP8XESMz84+Z+YeI2D0i/rkX+x4DPFaz3F4t6857gdtqlneJiLaI+GlEzK6W7QlsyMwXe7lPSZIkSVIv9aYP4psyc0PnQmY+Dby5F9tFF2VdtmeNiHcArcCCmuJ9q7dA/wb4ZLWpa1/2eW41YLatW7euF9WVJEmSpMbWm4A4LCJ27lyIiGZg5x7W79QOjK1ZbgHWbr1SRLwR+EdgVmY+31memWurjw8D3wOmURkcZ2REdDaN7XKf1e2uzczWzGwdNWpUL6orSZIkSY2tNwHxP4E7I+K9EfFe4A4qff+2ZSkwvjrq6HDgTGBR7QoRMQ34HJVw+ERN+e6doTQiXg0cATyQlRF17gZOr676LuCbvaiLJEmSJGkbejNIzZURsRJ4I5UmnrcD+/W8FWTmixFxAbAYGAZ8MTPvj4jLgLbMXESlSeluwNcr48/waGbOAl4LfC4iXqISYq+oGf30IuCGaj/I5cAX+nTGkiRJkqQubTMgVv0OeAmYAzwCfKM3G2XmfwP/vVXZJTXP39jNdj8GJnfz2sNURkiVJEmSJA2gbgNiRPw/VJqFngU8CdxIZZqLYwapbpIkSZKkQdTTHcRfAj8ETs7M1QAR8cFBqZWK4WT1kiRJknrQ0yA1b6XStPTuiPj3iDiOrqeZ0FCw8ib41oXwzGNAVh6/dWGlXJIkSZLoISBm5q2ZeQZwEJVpJj4I7BURn4mIEwapfhood14GGzu2LNvYUSmXJEmSJHoxzUVm/ikzv5qZJ1GZd3AFMK/wmmlgPdPet3JJkiRJDac38yBulplPZebnMvPYoiqkgoxo6Vu5JEmSpIbTp4CoIey4S6CpecuypuZKuSRJkiRhQGwcU+bAyZ+CEWOBqDye/ClHMZUkSZK0WU/TXKjeTJljIJQkSZLULe8gSpIkSZIAA6IkSZIkqcqAKEmSJEkCDIiSJEmSpCoDoiRJkiQJMCBKkiRJkqoMiJIkSZIkwIAoSZIkSaoyIEqSJEmSAAOiJEmSJKnKgChJkiRJAgyIkiRJkqQqA6IkSZIkCTAgSpIkSZKqCg2IEXFiRKyKiNURMa+L1z8UEQ9ExMqIuDMi9quWT42In0TE/dXXzqjZ5rqIeCQiVlR/phZ5DpIkSZLUKAoLiBExDLgGeBMwETgrIiZutdpyoDUzpwA3A1dWy58Fzs7M1wEnAp+MiJE1283NzKnVnxVFnYMkSZIkNZIi7yAeCqzOzIcz8wXgBuCU2hUy8+7MfLa6+FOgpVr+q8x8qPp8LfAEMKrAukqSJElSwysyII4BHqtZbq+Wdee9wG1bF0bEocBw4Nc1xR+rNj39RETsPBCVlSRJkqRGV2RAjC7KsssVI94BtAILtirfB/gP4N2Z+VK1+GLgIGA6sAdwUTf7PDci2iKibd26ddt3BpIkSZLUQIoMiO3A2JrlFmDt1itFxBuBfwRmZebzNeV/AXwb+Ehm/rSzPDMfz4rngS9Racr6Mpl5bWa2ZmbrqFG2TpUkSZKkbSkyIC4FxkfEuIgYDpwJLKpdISKmAZ+jEg6fqCkfDtwKfCUzv77VNvtUHwOYDdxX4DlIkiRJUsPYqagdZ+aLEXEBsBgYBnwxM++PiMuAtsxcRKVJ6W7A1yt5j0czcxYwB5gB7BkR51R3eU51xNKvRsQoKk1YVwDnFXUOkiRJktRIIrPLboF1pbW1Ndva2squhiRJkiSVIiKWZWbrttYrsompJEmSJGkIMSBKkiRJkgADoiRJkiSpyoAoSZIkSQIMiJIkSZKkKgOiJEmSJAkwIEqSJEmSqgyIkiRJkiTAgChJkiRJqjIgSpIkSZIAA6IkSZIkqcqAKEmSJEkCDIiSJEmSpCoDoiRJkiQJMCBKkiRJkqoMiJIkSZIkwIAoSZIkSaoyIEqSJEmSAAOiJEmSJKnKgChJkiRJAgyIkiRJkqQqA6IkSZIkCTAgSpIkSZKqDIiSJEmSJKDggBgRJ0bEqohYHRHzunj9QxHxQESsjIg7I2K/mtfeFREPVX/eVVP+VxFxb3Wfn4qIKPIcJEmSJKlRFBYQI2IYcA3wJmAicFZETNxqteVAa2ZOAW4GrqxuuwfwT8BhwKHAP0XE7tVtPgOcC4yv/pxY1DlIkiRJUiMp8g7iocDqzHw4M18AbgBOqV0hM+/OzGeriz8FWqrPZwJ3ZOZTmfk0cAdwYkTsA/xFZv4kMxP4CjC7wHOQJEmSpIZRZEAcAzxWs9xeLevOe4HbtrHtmOrz3u5TkiRJktRLOxW47676BmaXK0a8A2gFjtrGtn3Z57lUmqKy7777bquukiRJktTwiryD2A6MrVluAdZuvVJEvBH4R2BWZj6/jW3b+XMz1G73CZCZ12Zma2a2jho1artPQpIkSZIaRZEBcSkwPiLGRcRw4ExgUe0KETEN+ByVcPhEzUuLgRMiYvfq4DQnAIsz83HgDxHx+uropWcD3yzwHCRJkiSpYRTWxDQzX4yIC6iEvWHAFzPz/oi4DGjLzEXAAmA34OvV2SoezcxZmflURPy/VEImwGWZ+VT1+QeA64BmKn0Wb0OSJEmS1G9RGQy0vrW2tmZbW1vZ1ZAkSZKkUkTEssxs3dZ6RTYxlSRJkiQNIQZESZIkSRJgQJQkSZIkVRkQJUmSJEmAAVGSJEmSVGVAlCRJkiQBBkRJkiRJUpUBUZIkSZIEGBAlSZIkSVUGREmSJEkSYECUJEmSJFUZECVJkiRJgAFRkiRJklS1U9kVaEQLl69hweJVrN3QweiRzcydOYHZ08aUXS1JkqRSbNy4kfb2dp577rmyqyINebvssgstLS00NTVt1/YGxEG2cPkaLr7lXjo2bgJgzYYOLr7lXgBDoiRJakjt7e286lWvYv/99yciyq6ONGRlJk8++STt7e2MGzduu/ZhE9NBtmDxqs3hsFPHxk0sWLyqpBpJkiSV67nnnmPPPfc0HEr9FBHsueee/bobb0AcZGs3dPSpXJIkqREYDqWB0d9/SwbEQTZ6ZHOfyiVJklS897znPbzmNa9h0qRJ3a6TmVx44YUceOCBTJkyhZ///OebX/vyl7/M+PHjGT9+PF/+8pe3ebzvfe97nHTSSQNS9+1x3XXXsXbt2tKOP9Buv/12JkyYwIEHHsgVV1zR5Tof//jHmThxIlOmTOG4447jt7/97ebXhg0bxtSpU5k6dSqzZs3a5vF+85vf9HitFG3hwoU88MADhezbgDjI5s6cQHPTsC3KmpuGMXfmhJJqJEmSNLQsXL6GI664i3Hzvs0RV9zFwuVr+r3Pc845h9tvv73HdW677TYeeughHnroIa699lo+8IEPAPDUU09x6aWXcs899/Czn/2MSy+9lKeffnq76vHiiy9u13Z9VWpAXHkTfGISzB9ZeVx5U792t2nTJs4//3xuu+02HnjgAa6//vouw9O0adNoa2tj5cqVnH766Xz4wx/e/FpzczMrVqxgxYoVLFq0aLvrMlh/PwNiHZk9bQyXnzaZMSObCWDMyGYuP22yA9RIkiT1QueAf2s2dJD8ecC//obEGTNmsMcee/S4zje/+U3OPvtsIoLXv/71bNiwgccff5zFixdz/PHHs8cee7D77rtz/PHHdxk2b7/9dg466CCOPPJIbrnlls3l8+fP59xzz+WEE07g7LPP5rnnnuPd7343kydPZtq0adx9991AJdSdcsopnHjiiUyYMIFLL7108z4+/vGPM2nSJCZNmsQnP/lJ4OV3ua666irmz5/PzTffTFtbG29/+9uZOnUqHR2D2NVp5U3wrQvhmceArDx+68J+hcSf/exnHHjggRxwwAEMHz6cM888k29+85svW++YY45h1113BeD1r3897e3tfTrOsmXLOPjggzn88MO55pprNpdfd911vO1tb+Pkk0/mhBNOIDOZO3cukyZNYvLkydx4441A5a7xjBkzOPXUU5k4cSLnnXceL730EgDXX389kydPZtKkSVx00UWb973bbrttfn7zzTdzzjnn8OMf/5hFixYxd+5cpk6dyq9//es+nce2OIppCWZPG2MglCRJ2g49DfhX9OerNWvWMHbs2M3LLS0trFmzptvyWs899xzve9/7uOuuuzjwwAM544wztnh92bJlLFmyhObmZv71X/8VgHvvvZdf/vKXnHDCCfzqV78CKmHovvvuY9ddd2X69Om85S1vISL40pe+xD333ENmcthhh3HUUUex++67d3kep59+OldffTVXXXUVra2tA/K76bU7L4ONWwXSjR2V8ilztmuXXf3+77nnnh63+cIXvsCb3vSmzcvPPfccra2t7LTTTsybN4/Zs2e/bJt3v/vdfPrTn+aoo45i7ty5W7z2k5/8hJUrV7LHHnvwjW98gxUrVvCLX/yC9evXM336dGbMmAFU/n4PPPAA++23HyeeeCK33HILb3jDG7joootYtmwZu+++OyeccAILFy7ssg4Ab3jDG5g1axYnnXQSp59+eq9/T73lHURJkiQNGWUO+JeZLyuLiG7La/3yl79k3LhxjB8/nojgHe94xxavz5o1i+bmypgUS5Ys4Z3vfCcABx10EPvtt9/mgHj88cez55570tzczGmnncaSJUtYsmQJp556Kq985SvZbbfdOO200/jhD384IOc84J7p5q5dd+W90Jvff63//M//pK2tbYuQ9+ijj9LW1sbXvvY1/uEf/uFld+WeeeYZNmzYwFFHHQWw+e/TqfMOMlT+fmeddRbDhg1jr7324qijjmLp0qUAHHrooRxwwAEMGzaMs846iyVLlrB06VKOPvpoRo0axU477cTb3/52fvCDH2zfL2MAGBDLMMDtriVJkhpFmQP+tbS08Nhjj21ebm9vZ/To0d2Wb62n0PLKV75y8/OuAk93++guoALstNNOm5swAv2a+mDAjGjpW3kv9Pb3D/Dd736Xj33sYyxatIidd955c3nn+gcccABHH300y5cv32K7zBz0v9/W6w/W38+AONgKaHctSZLUKMoc8G/WrFl85StfITP56U9/yogRI9hnn32YOXMm3/nOd3j66ad5+umn+c53vsPMmTO32Paggw7ikUce2Xxn6vrrr+/2ODNmzOCrX/0qAL/61a949NFHmTChcn533HEHTz31FB0dHSxcuJAjjjiCGTNmsHDhQp599ln+9Kc/ceutt/LXf/3X7LXXXjzxxBM8+eSTPP/88/zXf/3X5mO86lWv4g9/+MNA/4q27bhLoGmrMN/UXCnfTtOnT+ehhx7ikUce4YUXXuCGG27ociTS5cuX8/73v59Fixbxmte8ZnP5008/zfPPPw/A+vXr+dGPfsTEiRO32HbkyJGMGDGCJUuWAGz++3RlxowZ3HjjjWzatIl169bxgx/8gEMPPRSoNDF95JFHeOmll7jxxhs58sgjOeyww/j+97/P+vXr2bRpE9dff/3mO5V77bUXDz74IC+99BK33nrr5mMU+fcrNCBGxIkRsSoiVkfEvC5enxERP4+IFyPi9JryYyJiRc3PcxExu/radRHxSM1rU4s8hwHXU7trSZIk9aioAf/OOussDj/8cFatWkVLSwtf+MIXAPjsZz/LZz/7WQDe/OY3c8ABB3DggQfyvve9j3/7t38DYI899uCjH/0o06dPZ/r06VxyySUvG/Bml1124dprr+Utb3kLRx55JPvtt1+3dfm7v/s7Nm3axOTJkznjjDO47rrrNt/tOvLII3nnO9/J1KlTeetb30prayuHHHII55xzDoceeiiHHXYYf/u3f8u0adNoamrikksu4bDDDuOkk07ioIMO2nyMc845h/POO2/wB6mZMgdO/hSMGAtE5fHkT213/0Oo3Cm9+uqrmTlzJq997WuZM2cOr3vd6wC45JJL2JfjjQAAB/dJREFUNo9KOnfuXP74xz/ytre9bYvpLB588EFaW1s5+OCDOeaYY5g3b97LAiLAl770Jc4//3wOP/zwzc2Bu3LqqacyZcoUDj74YI499liuvPJK9t57bwAOP/xw5s2bx6RJkxg3bhynnnoq++yzD5dffjnHHHMMBx98MIcccginnHIKAFdccQUnnXQSxx57LPvss8/mY5x55pksWLCAadOmDfggNdHTLc1+7ThiGPAr4HigHVgKnJWZD9Sssz/wF8D/BhZl5s1d7GcPYDXQkpnPRsR1wH91tW53Wltbs62tbftPZiDNHwl09TsPmL9hsGsjSZJUugcffJDXvva1ZVdjh3fdddfR1tbG1VdfXXZVtB2+973vcdVVV21xJ7coXf2biohlmbnNUYmKvIN4KLA6Mx/OzBeAG4BTalfIzN9k5krgpa52UHU6cFtmPltcVQdRAe2uJUmSJGkgFBkQxwCP1Sy3V8v66kxg60baH4uIlRHxiYjYuauNdlgFtLuWJElS/TvnnHO8eziEHX300YNy97C/igyIXQ3z06f2rBGxDzAZWFxTfDFwEDAd2AO4qItNiYhzI6ItItrWrVvXl8MWq4B215IkSZI0EHYqcN/twNia5RZgbR/3MQe4NTM3dhZk5uPVp89HxJeo9F98mcy8FrgWKn0Q+3jcYk2ZYyCUJEmqsa1pBCT1Tn/HmCnyDuJSYHxEjIuI4VSaii7q4z7OYqvmpdW7ikTlHWQ2cN8A1FWSJEkl2WWXXXjyySf7/cFWanSZyZNPPskuu+yy3fso7A5iZr4YERdQaR46DPhiZt4fEZcBbZm5KCKmA7cCuwMnR8Slmfk62DzC6Vjg+1vt+qsRMYpKE9YVwHlFnYMkSZKK19LSQnt7OztUtyBpiNpll11oadn+ATALm+ZiR7JDTXMhSZIkSYNsR5jmQpIkSZI0hBgQJUmSJEmAAVGSJEmSVNUQfRAjYh3w27Lr0YVXA+vLroTqlteXiuT1pSJ5faloXmMq0o56fe2XmaO2tVJDBMQdVUS09aajqLQ9vL5UJK8vFcnrS0XzGlORhvr1ZRNTSZIkSRJgQJQkSZIkVRkQy3Vt2RVQXfP6UpG8vlQkry8VzWtMRRrS15d9ECVJkiRJgHcQJUmSJElVBsQSRMSJEbEqIlZHxLyy66P6ExG/iYh7I2JFRLSVXR8NbRHxxYh4IiLuqynbIyLuiIiHqo+7l1lHDV3dXF/zI2JN9T1sRUS8ucw6auiKiLERcXdEPBgR90fE31fLfQ9Tv/VwfQ3p9zCbmA6yiBgG/Ao4HmgHlgJnZeYDpVZMdSUifgO0ZuaOOAePhpiImAH8EfhKZk6qll0JPJWZV1S/6No9My8qs54amrq5vuYDf8zMq8qsm4a+iNgH2Cczfx4RrwKWAbOBc/A9TP3Uw/U1hyH8HuYdxMF3KLA6Mx/OzBeAG4BTSq6TJHUrM38APLVV8SnAl6vPv0zlP0Spz7q5vqQBkZmPZ+bPq8//ADwIjMH3MA2AHq6vIc2AOPjGAI/VLLdTBxeSdjgJfCcilkXEuWVXRnVpr8x8HCr/QQKvKbk+qj8XRMTKahNUm/+p3yJif2AacA++h2mAbXV9wRB+DzMgDr7oosx2vhpoR2TmIcCbgPOrTbgkaaj4DPCXwFTgceBfy62OhrqI2A34BvAPmfn7suuj+tLF9TWk38MMiIOvHRhbs9wCrC2pLqpTmbm2+vgEcCuVps3SQPqfat+Lzj4YT5RcH9WRzPyfzNyUmS8B/47vYeqHiGii8uH9q5l5S7XY9zANiK6ur6H+HmZAHHxLgfERMS4ihgNnAotKrpPqSES8stpRmoh4JXACcF/PW0l9tgh4V/X5u4BvllgX1ZnOD+5Vp+J7mLZTRATwBeDBzPx4zUu+h6nfuru+hvp7mKOYlqA61O0ngWHAFzPzYyVXSXUkIg6gctcQYCfga15j6o+IuB44Gng18D/APwELgZuAfYFHgbdlpgONqM+6ub6OptI0K4HfAO/v7C8m9UVEHAn8ELgXeKla/H+o9BPzPUz90sP1dRZD+D3MgChJkiRJAmxiKkmSJEmqMiBKkiRJkgADoiRJkiSpyoAoSZIkSQIMiJIkSZKkKgOiJEm9FBGbImJFzc+8Adz3/hExpObKkiTVn53KroAkSUNIR2ZOLbsSkiQVxTuIkiT1U0T8JiL+JSJ+Vv05sFq+X0TcGRErq4/7Vsv3iohbI+IX1Z83VHc1LCL+PSLuj4jvRERzaSclSWpIBkRJknqveasmpmfUvPb7zDwUuBr4ZLXsauArmTkF+CrwqWr5p4DvZ+bBwCHA/dXy8cA1mfk6YAPw1oLPR5KkLURmll0HSZKGhIj4Y2bu1kX5b4BjM/PhiGgCfpeZe0bEemCfzNxYLX88M18dEeuAlsx8vmYf+wN3ZOb46vJFQFNm/nPxZyZJUoV3ECVJGhjZzfPu1unK8zXPN+FYAZKkQWZAlCRpYJxR8/iT6vMfA2dWn78dWFJ9fifwAYCIGBYRfzFYlZQkqSd+MylJUu81R8SKmuXbM7NzqoudI+IeKl++nlUtuxD4YkTMBdYB766W/z1wbUS8l8qdwg8Ajxdee0mStsE+iJIk9VO1D2JrZq4vuy6SJPWHTUwlSZIkSYB3ECVJkiRJVd5BlCRJkiQBBkRJkiRJUpUBUZIkSZIEGBAlSZIkSVUGREmSJEkSYECUJEmSJFX9X/N9AdkKoii+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1080x1080 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 2:\n",
    "Compare the validation and training accuracies with and without dropout -- what do your results suggest about dropout as a regularizer?\n",
    "\n",
    "## Answer:\n",
    "It will lower train accuracy,but improve val accuracy.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 3:\n",
    "Suppose we are training a deep fully-connected network for image classification, with dropout after hidden layers (parameterized by keep probability p). If we are concerned about overfitting, how should we modify p (if at all) when we decide to decrease the size of the hidden layers (that is, the number of nodes in each layer)?\n",
    "\n",
    "## Answer:\n",
    "Set p not too big or too small.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
